Determining qualified devices using zone information

ABSTRACT

Determining qualified devices using zone information is disclosed. Traffic data associated with the presence of a set of devices at a location is received. At least some of the devices included in the set are qualified as qualified devices. A set of sessions associated with at least some of the qualified devices is crated. Information associated with the set of sessions is provided as output.

CROSS REFERENCE TO OTHER APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.15/130,870, filed Apr. 15, 2016, which claims priority to U.S.Provisional Patent Application No. 62/191,270, filed Jul. 10, 2015, U.S.Provisional Patent Application No. 62/206,226, filed Aug. 17, 2015, U.S.Provisional Patent Application No. 62/222,046 filed Sep. 22, 2015, andU.S. Provisional Patent Application No. 62/249,934, filed Nov. 2, 2015all of which are incorporated herein by reference in their entireties.

BACKGROUND OF THE INVENTION

Technology is increasingly being used to track individuals as they visitretail shops and other locations. As one example, door counting devicescan be used by a retail store to track the number of visitors to aparticular store (e.g., entering through a particular door or set ofdoors) each day. As another example, in-store cameras can be used tomonitor the movements of visitors (e.g., observing whether they turnright or left after entering the store). A variety of drawbacks to usingsuch technologies exist. One drawback is cost: monitoring technology canbe expensive to install, maintain, and/or run. A second drawback is thatsuch technology is limited in the insight it can provide. For example,door counts do not distinguish between employees (who might enter andleave the building repeatedly during the course of the day) andshoppers. A third drawback is that such technology can be overlyinvasive. For example, shoppers may object to being constantlysurveilled by cameras—particularly when the cameras are used for reasonsother than providing security (e.g., assessing reactions to marketingdisplays).

BRIEF DESCRIPTION OF THE DRAWINGS

Various embodiments of the invention are disclosed in the followingdetailed description and the accompanying drawings.

FIG. 1A illustrates an example of an environment in which sensorscollect data from mobile electronic devices and the collected data isprocessed.

FIG. 1B depicts a graphical representation of example strengths anddurations and how classifications can be made.

FIG. 2 illustrates an embodiment of a traffic insight platform.

FIG. 3 illustrates a variety of example zoning rules and settings.

FIG. 4A illustrates an example of a zoning metric table.

FIG. 4B illustrates an example of a zoning metric table.

FIG. 4C illustrates an example of a zoning metric table.

FIG. 5 illustrates an embodiment of a process for determining qualifieddevices using zone information.

FIGS. 6-8 show interfaces depicting zoning information for a nationalretailer at a particular location in Boston.

FIGS. 9-15 show interfaces depicting zoning information for an airport.

FIGS. 16A and 16B show interfaces depicting zoning information for ahotel.

FIGS. 17-20 show examples of interfaces for creating an event.

FIGS. 21-22 show examples of event summary page interfaces.

FIG. 23 shows an example of an interface depicting loyalty information.

FIG. 24 shows an example of an interface in which a comparison betweentwo periods' re-engagement is displayed.

FIG. 25 shows an example of an interface in which options for includingvisitor loyalty data in a dashboard view is displayed.

FIG. 26 illustrates an embodiment of a process for assessing visitorcomposition.

FIG. 27-30 depict an example implementation of an events pipelinewrapper script.

FIG. 31 depicts sample data from an event frequency table.

FIG. 32 illustrates an embodiment of a process for determining co-visitsby visitors.

FIG. 33 illustrates an embodiment of a process for determiningre-visitation by visitors.

FIG. 34 illustrates an embodiment of a process for assessing visitorfrequency during an event.

DETAILED DESCRIPTION

The invention can be implemented in numerous ways, including as aprocess; an apparatus; a system; a composition of matter; a computerprogram product embodied on a computer readable storage medium; and/or aprocessor, such as a processor configured to execute instructions storedon and/or provided by a memory coupled to the processor. In thisspecification, these implementations, or any other form that theinvention may take, may be referred to as techniques. In general, theorder of the steps of disclosed processes may be altered within thescope of the invention. Unless stated otherwise, a component such as aprocessor or a memory described as being configured to perform a taskmay be implemented as a general component that is temporarily configuredto perform the task at a given time or a specific component that ismanufactured to perform the task. As used herein, the term ‘processor’refers to one or more devices, circuits, and/or processing coresconfigured to process data, such as computer program instructions.

A detailed description of one or more embodiments of the invention isprovided below along with accompanying figures that illustrate theprinciples of the invention. The invention is described in connectionwith such embodiments, but the invention is not limited to anyembodiment. The scope of the invention is limited only by the claims andthe invention encompasses numerous alternatives, modifications andequivalents. Numerous specific details are set forth in the followingdescription in order to provide a thorough understanding of theinvention. These details are provided for the purpose of example and theinvention may be practiced according to the claims without some or allof these specific details. For the purpose of clarity, technicalmaterial that is known in the technical fields related to the inventionhas not been described in detail so that the invention is notunnecessarily obscured.

Individuals increasingly carry mobile electronic devices (e.g., mobilephones, laptops, tablets, etc.) virtually all of the time as they goabout their daily lives. Using techniques described herein, a variety ofsensors can be used to detect the presence of such devices (e.g.,devices with WiFi, cellular, and/or Bluetooth capabilities) based on thecapabilities of the sensors. And, insights about the individualscarrying those devices can be gained.

Throughout the Specification, the primary example of a “sensor” is aWiFi access point, and the primary example of a mobile electronic deviceis a cellular phone with WiFi enabled (though not necessarily associatedwith the “observing” WiFi access point). It is to be understood that thetechniques described herein can be used in conjunction with a variety ofkinds of sensors/devices, and the techniques described herein adapted asapplicable. For example, in addition to WiFi access points, RadioFrequency (RF) receivers that detect RF signals produced by cellularphones, and Bluetooth receivers that detect signals produced byBluetooth capable devices can be used in accordance with techniquesdescribed herein. Further, a single device can have multiple kinds ofsignals detected and used in accordance with techniques describedherein. For example, a cellular phone may be substantiallysimultaneously detected by one or more sensors through a WiFiconnection, a cellular connection, and/or a Bluetooth connection, and/orother wireless technology present on a commodity cellular phone. Datacollected by the sensors can be used in a variety of ways, and a varietyof insights can be gained (e.g., about the individuals carrying thedevices). As will be described in more detail below, the data can becollected in efficient and privacy preserving ways.

FIG. 1A illustrates an example of an environment in which sensorscollect data from mobile electronic devices and the collected data isprocessed. In the example shown, Alice and Bob are present in a retailspace 102. In particular, Alice and Bob are both shoppers shopping at abrick-and-mortar clothing store (hereinafter “ACME Clothing”). Includedin retail space 102 are a set of sensors (104-108). Sensors 104-108 areWiFi access points (e.g., offering WiFi service to customers and/orproviding service to point-of-sales and other store infrastructure).Sensors 104-108 each detect wireless signals from mobile electronicdevices. In the example shown in FIG. 1A, Alice and Bob each carry amobile device (e.g., cellular phones 110 and 112, respectively).

Also included in the environment shown in FIG. 1A is an airport space150. Charlie and Dave are passengers in airport space 150, and Eve is anemployee at a bookstand. Charlie, Dave, and Eve each carry respectivemobile devices 152-156. Sensors, including sensors 158-164 are presentin airport space 150.

The sensors depicted in FIG. 1A (i.e., sensors 104-108 and 158-164) arecommodity WiFi access points. Other sensors can also be used inconjunction with techniques described herein as applicable. As will bedescribed in more detail below, the sensors included in spaces 102 and150 can be grouped into zones (an arbitrary collection of sensors). Forexample, suppose retail space 102 is a two story building, with sensors108 and 110 on the first floor, and sensor 106 on the second floor.Sensors 108 and 110 can be grouped into a “First Floor” zone, and Sensor106 can be the sole sensor placed in a “Second Floor” zone.

Floors are one example of zoning, and tend to work well in retailenvironments (e.g., due to WiFi resolution of approximately 10 meters).Other segmentations can also be used for zoning (including in retailenvironments), depending on factors such as wall placement, asapplicable. As another example, airport space 150 might have severalzones, corresponding to areas such as “Ticketing,” “A Gates,” “B Gates,”“Pre-Security Shops,” “A Gate Security,” “Taxis,” etc. Further, thezones can be arranged in a hierarchy. Using airport space 150 as anexample, two hierarchical zones could be: Airport-Terminal 1-A Gates andAirport-Terminal 2-Pre-Security Shops.

As will be described in more detail below, signal strength and signalduration can be used to classify devices observed by a sensor. FIG. 1Bdepicts a graphical representation of example strengths and durationsand how classifications can be made. Signal strength can be used as anindicator of whether an observed device is within the geographicconfines of a sensor's zone. In some embodiments, if the device isdetermined to be within the geographic boundaries of the sensor's zone,it is classified as a visitor. If the signal is weak enough that it isdetermined to be outside the boundaries of the sensor's zone, it isdetermined to be a walk-by. If a zone has more than one sensor, multiplesensor readings can be used to determine if a device is a visitor or awalk-by. Certain devices can also be determined to be access points orother devices that do not belong to visitors or walk-bys, as illustratedin FIG. 1B. By measuring the length of time that the device is seen, forexample, a determination can be made (e.g., probabilistically) whether adevice belongs to staff, happens to be an access point inside the zone,and/or is otherwise a device type that should be ignored (e.g., aprinter or point-of-sales terminal).

Onboarding

In the following discussion, suppose a representative of ACME Clothingwould like to gain insight about shopper traffic in the store. Examplesof information ACME Clothing would like to learn include how manyshoppers visit the second floor of the store in a given day, how muchtotal time shoppers spend in the store, and how much time they spend onthe respective floors of the store. Using techniques described herein,ACME Clothing can leverage commodity WiFi access points to learn theanswers to those and other questions. In particular, in variousembodiments, ACME Clothing can leverage the access points that itpreviously installed (e.g., to provide WiFi to shoppers and/orstaff/sales infrastructure) without having to purchase new hardware.

In various embodiments, ACME Clothing begins using the services oftraffic insight platform 170 as follows. First, a representative of ACMEClothing (e.g., via computer 172) creates an account on platform 170 onbehalf of ACME Clothing (e.g., via a web interface 174 to platform 170).ACME Clothing is assigned an identifier on platform 170 and a variety oftables (described in more detail below) are initialized on behalf ofACME Clothing.

A first table (e.g., a MySQL table), referred to herein as an “assettable,” stores information about ACME Clothing and its sensors. Theasset table can be stored in a variety of resources made available byplatform 170, such as relational database system (RDS) 242. To populatethe table, the ACME representative (hereinafter referred to as Rachel)is prompted to provide information about the access points present inspace 102, such as their Media Access Control (MAC) addresses, and, asapplicable, vendor/model number information. Rachel is also asked tooptionally provide grouping information (e.g., as applicable, toindicate that sensors 108 and 110 are in a “First Floor” group and 112is in a “Second Floor group). The access point information can beprovided in a variety of ways. As one example, Rachel can be asked tocomplete a web form soliciting such information (e.g., served byinterface 174). Rachel can also be asked to upload a spreadsheet orother file/data structure to platform 170 that includes the requiredinformation. The spreadsheet (or portions thereof) can be created byRachel (or another representative of ACME Clothing) or, as applicable,can also be created by networking hardware or other third party tools.Additional (optional) information can also be included in the assettable (or otherwise associated with ACME Clothing's account). Forexample, a street address of the store location, city/state informationfor the location, time-zone information for the location, and/orlatitude/longitude information can be included, along withend-user-friendly descriptions (e.g., providing more information aboutthe zones, such as that the “Zone 1” portion of ACME includes shoes andaccessories, and that “Zone 2” includes outerwear).

The zoning hierarchy framework is flexible and can easily be modified byRachel, as needed. For example, after an initial set up ACME Clothing'szones, Rachel can split a given zone into pieces, or combine zonestogether (reassigning sensors to the revised zones as applicable, addingnew sensors, etc.). The asset table on platform 170 will be updated inresponse to Rachel's modifications.

In some embodiments, Rachel is asked to provide MAC addresses (or otheridentifiers) of known non-visitor devices. For example, Rachel canprovide the identifiers of various computing equipment present in space102 (e.g., printers, copiers, point of sales terminals, etc.) to ensurethat they are not inadvertently treated by platform 170 as belonging tovisitors. As another example, Rachel can provide the identifiers ofstaff-owned mobile computing devices (and designate them as belonging tostaff, and/or designate them as to be ignored, as applicable). As willbe described in more detail below, Rachel need not supply such MACaddresses, and platform 170 can programmatically identify devices thatare probabilistically unlikely to belong to visitors and exclude themfrom analysis as applicable.

In the example of FIG. 1A, ACME Clothing is a single location business.Techniques describes herein can also be used in conjunction withmulti-location businesses. In such a scenario, additional hierarchicalinformation can be provided during onboarding. As one example, a retailstore with 50 locations could organize its access points intogeographical or other regions (e.g., with West Coast—California—Store123—First Floor—AA:12:34:56:78:FF and West Coast—Nevada—Store 456—SecondFloor—BB:12:34:56:67:FF being two examples of information supplied toplatform 170 about two sensors). In some cases, a parent company may ownstores of multiple brands. For example, Beta Holding Company may ownboth “Beta Electronics Retail” and “Delta Electronics Depot.” The assetstable for Beta Holding Company can accordingly include the respectivebrand names in the hierarchy of access points if desired (e.g., “BetaHolding Company—Beta Electronics Retail—California—Store 567 . . . ” and“Beta Holding Company—Delta Electronics Depot—Texas—Store 121 . . . ”).

Ingesting Sensor Data

Rachel is provided (e.g., via interface 174) with instructions forconfiguring sensors 104-108 to provide platform 170 with data that theycollect. Typically, the collected data will include the MAC addressesand signal strength indicators of mobile devices observed by thesensors, as well as applicable timestamps (e.g., time/duration ofdetection), and the MAC address of the sensor that observed the mobiledevice. For some integrations, the information is sent in JSON using anexisting Application Programming Interface (API) (e.g., by directing thehardware to send reporting data to a particular reporting URL, such ashttp://ingest.euclidmetrics.com/ACMEClothing or hardware vendor tailoredURLs, such as http://cisco.ingest.euclidmetrics.com orhp.ingest.euclidmetrics.com, as applicable, where the data is providedin different formats by different hardware vendors). Accordingly, theconfiguration instructions provided to Rachel may vary based on whichparticular hardware (e.g., which manufacturer/vendor of commodity accesspoint) is in use in retail space 102. For example, in some cases, thesensors may report data directly to platform 170 (e.g., as occurs withsensors 104-108). In other cases, the sensors may report data to acontroller which in turn provides the data to platform 170 (e.g., asoccurs with sensors 158-164 reporting to controller 166).

In the example environment shown in FIG. 1A, and in FIG. 2, platform 170is implemented using cloud computing resources, such as Amazon WebServices (AWS) Google Cloud, or Microsoft's Azure. Resources describedherein (or portions thereof) can also be provided by dedicated hardware(e.g., operated by an entity on behalf of itself, such as a governmentalentity). Whenever platform 170 is described as performing a task, asingle component, a subset of components, or all components of platform170 may cooperate to perform the task. Similarly, whenever a componentof platform 170 is described as performing a task, a subcomponent mayperform the task and/or the component may perform the task inconjunction with other components. Various logical components and/orfeatures of platform 170 may be omitted and the techniques describedherein adapted accordingly. Similarly, additional logicalcomponents/features can be added to appliance 170 as applicable.

As shoppers, such as Alice and Bob, walk around in retail space 102,data about the presence of their devices (110 and 112) is observed bysensors (e.g., sensors 104-108) and reported to platform 170. Forexample, the MAC addresses of devices 110/112, and their observed signalstrengths are reported by the observing sensors. The ingestion of thatdata will now be described, in conjunction with FIG. 2.

FIG. 2 illustrates an embodiment of a traffic insight platform, such asplatform 170. Platform 170 receives data 202 (via one or more APIs) intoan AWS elastic cloud load balancer (204), which splits the ingestioninfrastructure across multiple EC2 instances (e.g., ingestors 206-210).The ingestors create objects out of the received data, which areultimately written (e.g., as JSON) to disk (e.g., as hourly writes toS3) 212 and a real time messaging bus (e.g., Apache Kafka).

The ingestors are built to handle concurrent data ingestion (e.g., usingScala-based spray and Akka). As mentioned above, data provided bycustomers such as ACME Clothing typically arrives as JSON, though theformatting of individual payloads may vary between customers of platform170. As applicable, ingestors 206-210 can rewrite the received data intoa canonical format (if the data is not already provided in that format).For example, in various embodiments, ingestors 206-210 include a set ofparsers specific to each customer and tailored to the sensor hardwardemanufacturer(s) used by that customer (e.g., Cisco, Meraki, Xirrus,etc.). The parsers parse the data provided by customers and normalizethe data in accordance with a canonical format. In various embodiments,additional processing is performed by the ingestors. In particular, thereceived MAC addresses of mobile devices are hashed (e.g., for privacyreasons) and, in some embodiments, compared against a list of opted-outMAC addresses. Additional transformations can also be performed. Forexample, in addition to hashing the MAC address, a daily seed can beused (e.g., a daily seed used for all hashing operations for a 24-hourperiod), so that two different hashes will be generated for the samedevice if it is seen on two different days. If data is received for aMAC that has opted-out, the data is dropped (e.g., not processedfurther). One way that users can opt-out of having their data processedby platform 170 is to register the MAC addresses of their mobile deviceswith platform 170 (e.g., using a web or other interface made availableby platform 107 and/or a third party).

As a given ingestor processes the data it has received, it writes to alocal text log. Two example log lines written by an ingestor instance(e.g., ingetstor 206) and in JSON are as follows:

Apr. 8, 2015 4:00:00 PM org.apache.jsp.index_jsp_jspService

INFO:{“sn”:“40:18:B1:38:7A:40”,“pf”:1,“ht”:[{“sl”:−89,“ot”:1396972150,“s2”:46122,“is”:667,“sm”:“88329B”,“so”:−89,“sc”:−89,“il”:0,“sh”:−86,“ct”:1396972151,“si”:“b533c82bfeef4232”,“ih”:624,“ap”:0,“cn”:6,“ss”:−526,“cf”:5180,“i3”:243039545,“s3”:−4044994,“i2”: 391057}],“tp”:“ht”,“sq”:846077,“vs”:3}

Apr. 8, 2015 4:00:00 PM org.apache.jsp.index_jsp_jspService

INFO:{“sn”:“40:18:B1:39:32:C0”,“pf”:1,“ht”:[{“sl”:−68,“ot”:1396972136,“s2”:54162,“is”:1285,“sm”:“68A86D”,“so”:−53,“sc”:−61,“il”:20,“sh”:−52,“ct”:1396972138,“si”:“2e5e1d2807e5d3ad”,“ih”:604,“ap”:0,“cn”:15,“ss”:−898,“cf”:2437,“i3”:226673720,“s3”:−3290416,“i2”: 420062}],“tp”:“ht”,“sq”:830438,“vs”:3}

In the above example log lines, “sn” is a serial number (or) MAC of thesensor that observed a mobile device (i.e., that has transmitted thereporting data to platform 107, whether directly or through acontroller). The “pf” is an identifier of the customer sending the data.The “ht” is an array of detected devices, and includes the following:

sl: minimum signal strength

ot: timestamp of first frame (unix time in seconds)

s2: sum of the signal strength squared (to calculate variance)

is: sum of intervals (in seconds)

sm: station organizationally unique identifier or manufactureridentifier

so: first signal strength detected

sc: last signal strength detected

il: minimum interval (in seconds)

sh: maximum signal strength

ct: timestamp of last frame (unix time in seconds)

si: station identifier/detected device identifier, hashed

ih: maximum interval (in seconds)

ap: a flag indicating whether the reporting sensor is an access point ornot

cn: count of number of frames summarized in this message for this device

ss: summation of signal strength (a negative number)

cf: frequency last frame received on

i3: sum of interval cubed

s3: sum of signal strength cubed (to calculate skew)

i2: sum of interval squared

The “tp” value indicates the type of message (where “ht” is a hit—adevice being seen by the sensor, and “hl” is a health message—a ping thesensor sends during periods of inactivity). The “sq” value is a sequencenumber—a running count of messages from the sensor (and, in someembodiments, resets to zero if the sensor reboots). The “vs” value is aversion number for the sensor message.

Once an hour, a script (e.g., executing on ingestor 206) gzips the localingestor log and pushes it to an S3 bucket. The other ingestors (e.g.,ingestor 208 and 210) similarly provide gzipped hourly logs to the S3bucket, where they will be operated on collectively. The logs stored inS3 are loaded (e.g., by a job executing on the S3 bucket) into MySQL andRedshift, which is in turn used by metrics pipeline 230.

Further, as the ingestors are writing their local logs, threads on eachof the ingestors (e.g., Kafka readers) tail the logs and provide the logdata to a Kafka bus for realtime analysis (described in more detailbelow) on an EC2 instance.

Zoning Pipeline

A variety of jobs execute on platform 170. Zoning-related jobs arerepresented in FIG. 2 as “zoning pipeline” 216. Various portions of thezoning pipeline are written in scripting languages (e.g., as pythonscripts) or written using S3 tools, etc., as applicable. The zoningpipeline is collectively executed by a cluster of EC2 instances workingin parallel (e.g., using a Map Reduce framework) and runs as a batch job(e.g., runs once a day). Other pipelines described herein (e.g.,realtime pipeline 226 and metrics pipeline 230) are similarlycollections of scripts collectively executed by a cluster of EC2instances.

Extract from S3

Each day (or another unit of time, as applicable, in alternateembodiments), the following occurs on platform 170. In a first stage,“Extract from S3” (218) the zoning pipeline reads the logs (provided byingestors 206-210) stored in an S3 bucket the previous day. A “metadatajoin” script executes, which annotates the log lines with additional(e.g., human friendly) metadata. As one example, during the execution ofthe metadata join, the MAC address of a reporting sensor (included inthe log data) is looked up (e.g., in an asset table) and informationsuch as the human friendly name of the owner of the sensor (e.g., “ACMEClothing”), the human friendly location (e.g., “SF Store” or “Store 123,the hierarchy path (as applicable), etc. are annotated into the loglines. Minute-level aggregation is also performed, using the first seen,last seen, and max signal strength values for a given minute for a givendevice at a given sensor to collapse multiple lines (if present for adevice-sensor combination) into a single line. So, for example, ifsensor 108 has made six reports (in a one minute time interval) that ithas seen device 122, during minute level aggregation, the six linesreported by sensor 108 are aggregated into a single line, using thestrongest maximum signal strength value.

The output of the “Extract from S3” process (annotated log lines,aggregated at the minute level) is written to a new S3 bucket foradditional processing. As used hereinafter, the newly written logs(i.e., the output of “Extract from S3”) is a daily set of “annotatedlogs.”

Zoning Classification

The next stage of the zoning pipeline makes a probabilisticdetermination of whether a given mobile electronic device for which datahas been received (e.g., by platform 170 from retail space 102) belongsto a shopper (or, in other contexts, such as airport space 150, otherkinds of visitors, such as passengers) or represents a device thatshould (potentially) be excluded from additional processing (e.g., onebelonging to a store employee, a point-of-sale terminal, etc.). Thefiltering determination (e.g., “is visitor” or not) is made using avariety of features/parameters, described in more detail below. Thedetermination is described herein as being made by a “zoning classifier”(222) which is a piece of zoning pipeline 216 (i.e., is implementedusing a variety of scripts collectively executing on a cluster of EC2instances, as with the rest of the zoning pipeline).

During processing of the most recently received daily log data (i.e.,the most recently processed annotated logs), zoning classifier 222groups that daily log data by device MAC. For example, all of Alice'sdevice 110 log entries are grouped together, and all of Bob's device 112log entries are grouped together. The grouped entries are sorted bytimestamp (e.g., with Alice's device 110's first time stamp appearingfirst, and then its second time stamp appearing next, etc.). In variousembodiments, a decision tree of rules is used to filter devices. In someembodiments, at each level, the tree branches, and non-visitor devicesare filtered out. One example of a filtering rule is the Boolean, “tooshort.” This Boolean can be appended to any device seen for less thanthirty seconds, for example. The “too short” Boolean is indicative of awalk-by—someone who didn't linger long enough to be considered avisitor. A second example of a filtering rule is the Boolean, “toolong,” which is indicative of a “robot” device (i.e., not a personaldevice carried by a human). This Boolean can be appended to any device(e.g., a cash machine, printer, point of sale terminal, etc.) that isseen for more than twenty hours in a given day, for example.

More complex filtering rules can also be employed. As one example,suppose Eve (an employee at a bookstand in airport space 150) has apersonal cellular phone 156. On a given day (e.g., where Eve works afour hour shift), Eve's device 156 might appear to be similar to apassenger's device (e.g., seen in various locations within the airportover a four hour period of time). However, by examining a moving ten-daywindow of annotated log data, Eve's device can be filtered fromconsideration of belonging to a customer. Accordingly, in variousembodiments, zoning classifier 222 reads the last ten days (or anotherappropriate length of time) of annotated logs into RAM, and providesfurther annotations (e.g., as features) appended to each row of theannotated logs stored in RAM. As one example, a feature of “how manydays seen” can be determined by examining the last ten day of annotatedlog data, and a value (e.g. “2” days or “3” days, etc.) associated witha given device, as applicable, and persisted in memory. Further, if thenumber of days exceeds a threshold (three days or more), an additionalfeature “exhibits employee-like behavior” can be associated with Eve'sdevice. Another feature, “seen yesterday” can similarly be determinedused to differentiate visitors from employees.

Example rules and settings for a variety of kinds of customers are shownin FIG. 3. Rules (and threshold values, also referred to herein asparameters) can be customized based on customer type/customer needs(e.g., via interface 174), and form a “zoning” model for each location.As one example, one filtering rule that can be used is “seen withinhours of operation” (the hours of which will vary based on customer, andcan be defined as a parameter, e.g., by an employee like Rachel).Similarly, while a single retail example is shown in FIG. 3, differentretail environments can specify different parameters/thresholds forthose features as applicable. For example, parameters applicable to aboutique clothing store on Rodeo Drive (with too short=30 seconds orrepeat visits in ten days >2 being indicative of an employee device) maybe different from those applicable to a grocery store in Topeka (withtoo short=120 seconds or repeat visitors in ten days >4 being indicativeof an employee device). Some features may have binary parametersindicative of whether or not a device is a visitor or not. For example,if a device is flagged as being observed “too long,” a zoning model canuse that information to conclude that the device is not a visitor. Otherfeatures may have varying weights assigned to them, and thedetermination of whether a device is a visitor or not may be madedependant on the combination of features observed (and the weightsassigned). For example, a high number of repeat visits to a coffee shop,while indicative of an employee device, could also plausibly be a loyalcustomer device. Accordingly, a zoning model for the coffee shop mayweight repeat visits as being less probative of whether a device belongsto a customer or not. In various embodiments, platform 170 makesavailable a variety of default zoning models (e.g.: hotel, indoorshopping mall, outdoor shopping mall, etc.) which can be customized asapplicable (e.g., by a user of computer 172 via interface 174).

An example of a device which could survive a filtering decision tree isone that is seen more than 30 seconds, seen fewer than five hours, has areceived signal strength indicator (RSSI) of at least 50, and is notseen more than twice in the last ten days. Such a device isprobabilistically likely to be a visitor. Devices which are not filteredout are labeled with a Boolean flag of “is visitor” and processing onthe data for those devices continues. In various embodiments, theannotated log data for the day being operated on (i.e., for whichmetrics, described in more detail below, are calculated) is referred toas a “qualified log” once employee/printer/etc. devices have beenremoved and only those devices probabilistically corresponding tovisitors remain. The next stage of classification is to determine“sessions” using the qualified log lines.

As used herein, a “pre-session” is a set of qualified log lines (for agiven mobile electronic device) that split on a gap of 30 or minutes. Apre-session is an intermediate output of the zoning classifier. SupposeAlice's device 110 is observed (e.g., by sensor 108) for fifteenminutes, starting at 13:01 on Monday. The annotated log contains fifteenentries for Alice (due to the minute-level aggregation described above).The zoning classifier generates a pre-session for Alice, which groupsthese fifteen entries together. Suppose Bob's device 112 is observed(e.g., by sensor 108) for two minutes, then is not observed for an hour,and then is seen again for an additional ten minutes on Monday. Thezoning classifier will generate two pre-sessions for Bob because thereis a one hour gap (i.e., more than 30 minute gap) between times thatBob's device 112 was observed. The first pre-session covers the twominute period, and the second pre-session covers the ten minute period.As yet another example, if Charlie's device 152 is observed for fourconsecutive hours on a Wednesday, Charlie will have a single pre-sessioncovering the four-hour block of annotated logs pertinent to his device'spresence being detected in airport space 150.

In some cases, a pre-session may include data from only a single sensor.As one example, suppose Alice is on the second floor of retail space 102(which only includes a single access point, sensor 106). Alice'spre-session might accordingly only include observations made by sensor106. In other cases, a pre-session may include data from multiplesensors. As one example, suppose Charlie (a passenger) arrives atairport space 150, checks in for his flight (in the Ticketing area),purchases a magazine at a pre-security shop, proceeds through security,and then walks to his gate (e.g., gate A15). Charlie is present inairport space 150 for four hours, and his device 152 is observed byseveral sensors during his time in airport space 150. As mentionedabove, Charlie's pre-session is (in this example) four hours long. Insome cases, a single sensor may have observed Charlie during a givenminute. For example, when Charlie first arrives at airport space 150,his device 152 is observed by a sensor (158) located in the Ticketingarea for a few minutes. Once he is checked in, and he walks toward thepre-security shopping area, his device 152 is observed by both theTicketing area sensor (158) and a sensor (162) located in thepre-security shopping area for a few minutes. Suppose, for example,twenty minutes into Charlie's presence in airport space 150, device 152is observed by both sensor 158 (strongly) and sensor 162 (weakly). AsCharlie gets closer to the stores, the signal strength reported withrespect to his device will become weaker with respect to sensor 158 andstronger with respect to sensor 162. In various embodiments, theclassifier examines each minute of a pre-session, and, where multipleentries are present (i.e., a given device was observed by multiplesensors), the classifier selects as representative the sensor whichreported the strongest signal strength with respect to the device. Avariety of values can be used to determine which sensor reported thestrongest signal strength for a given interval. As one example, the maxsignal strength value (“sh”) can be used. In various embodiments, thisreduction in log data being considered is performed earlier (e.g.,during minute level aggregation), or is omitted, as applicable.

Next, a zone mapper 224 (another script or set of scripts operating aspart of zoning pipeline 216) annotates each line of each pre-session andappends the zone associated with the observing sensor (or sensor whichhad the strongest signal strength, as applicable). Returning to theexample of Charlie walking around inside airport space 150, thefollowing is a simplified listing of a portion of log data associatedwith Charlie's device 152. In particular, the simplified data shows atimestamp and an observing sensor:

09:50—AP4

. . .

10:00—AP4

10:01—AP4

10:02—AP2

10:03—AP1

10:04—AP3

10:05—AP2

. . .

10:15—AP2

Suppose AP1, AP2, and AP3 are each sensors present in the “A Gates”section of airport space 150, and AP4 is a sensor present in thesecurity checkpoint area. The zone mapper annotates Charlie's log dataas follows:

09:50—AP4—Security

. . .

10:00—AP4—Security

10:01—AP4—Security

10:02—AP2—A-Gates

10:03—AP1—A-Gates

10:04—AP3—A-Gates

10:05—AP2—A-Gates

. . .

10:15—AP2—A-Gates

The Zone mapper then collapses contiguous minutes in which the devicewas seen in the same zone into a single object (referred to herein as a“session”), which can then be stored and/or used for further analysis asdescribed in more detail below. A device level “session,” labeled by azone, is the output of the classification process. In variousembodiments, the session object includes all (or portions of) theannotations made by the various stages of the zoning pipeline. In theexample of Charlie, the excerpts above indicate that he spent twelveminutes in the security area (from 9:50-10:01) and fourteen minutes inthe A-Gates area (10:02-10:15). Two sessions for Charlie will be stored(e.g., in a MySQL database /S3 or other appropriate storage): onecorresponding to his twelve minutes in security, and one correspondingto his fourteen minutes in security, along with additional data, asapplicable.

Realtime Pipeline

Returning to FIG. 2, as previously mentioned, as ingestors 206-210 writetheir local logs, threads on each of the ingestors (e.g., Kafka readers)tail the logs and provide the log data to a Kafka bus for realtimeanalysis on an EC2 instance. As a data source, S3 is inexpensive andreasonably fast. Kafka is more expensive, but significantly faster.

Realtime pipeline 226 operates in a similar manner to zoning pipeline216 except that it works on a smaller time scale (and thus with lessdata). For example, instead of operating on ten days of historical data,in various embodiments, the realtime pipeline is configured to examinean hour of historical data. And, where the zoning pipeline executes as adaily batch operation, the realtime pipeline batch operation occursevery five minutes. And, instead of writing results to S3, the realtimepipeline writes to Cassandra (228) tables, which are optimized forparallel reads and writes. The realtime pipeline 226 also accumulatesthe qualified log data. In some embodiments, a list of banned devices isheld in memory, where the devices included on that list are selectedbased on being seen “too long.” Such devices (e.g., noisy devicespinging every two seconds for 20 hours) might be responsible for 60-80%of traffic, and excluding them will make the realtime processing moreefficient.

As will be described in more detail below, metrics generated withrespect to zoning pipeline data will typically be consumed via reports(e.g., served via interface 174 to an administrator, such as one usingcomputer 172). Metrics generated with respect to realtime pipeline dataare, in various embodiments, displayed on television screens (e.g.,within airport space 150) or otherwise made publicly available (e.g.,published to a website), as indicators of wait times, and refreshfrequently (e.g., once a minute). In some embodiments, realtime data canbe used to trigger email or other messages. For example, suppose a givencheckpoint at a particular time of day typically has a wait time ofapproximately five minutes (and a total number of five to ten peoplewaiting in line). If the current wait time is twenty minutes and/orthere are fifty people in line (e.g., as determined by realtime pipeline226), platform 170 can output a report (e.g., send an email, an SMS, orother message) to a designated recipient or set of recipients, allowingfor the potential remediation of the congestion.

Realtime analysis using the techniques described herein is particularlyuseful for understanding wait times (e.g., in security, in taxi lines,etc.) and processes such as hotel check-in/check-out. An example use ofanalysis performed using the zoning techniques described herein isdetermining how visitors move through a space. For example, historicalanalysis can be used to determine where to place items/workers/etc.based on flow.

Zoning/Realtime Metrics

Platform 170 includes a metrics pipeline (230) that generates metricsfrom the output of the zoning pipeline (and/or realtime pipeline asapplicable). Various metrics are calculated on a recurring basis (e.g.,number of visitors per zone per hour) and stored (e.g., in RedShiftstore 236). In various embodiments, platform 170 uses a lambdaarchitecture for the metrics pipeline (and other pipelines, asapplicable). One example implementation of metrics pipeline 230 is aSpark cluster (running in Apache Mesos). In the case of realtime metricsgeneration (e.g., updating current security line and/or taxi line waittimes), analysis is performed using a Spark Streaming application (234),which stores results in Cassandra (228) for publishing.

Summaries used to generate reports 232 (made available to end users viaone or more APIs provided by platform 170) are stored in MySQL. Suchstored metrics will include a time period, a zone, and a metric namevalue. Sample zoning metric tables are shown in FIGS. 4A-4C. Inparticular, Table 4A holds metrics about visits and durations in thedaily/hourly/15-minute level. Table 4B holds a histogram of durationtimes: within a given time period in a given location, how many visitorswere around for 0-10, 11-20, 21-30, 31-40, and more than 41 minutes.Table 4C holds conditional metrics looking at the device level: apairwise examination of different zones—of the people seen in one zone,what percentage of them were also seen at another zone. Additionalmetrics can also be determined and are described in more detail below.

Reporting data 232 is made available to representatives of customers ofplatform 170 (e.g., Rachel) via interface 174. As another example,reporting data 232 is made available to airport space 150 visitors(e.g., via television monitors, mobile applications, and/or websitewidgets), reflecting information such as current wait times.

For metrics calculated on an hourly basis, any sessions that do notinclude that time period are ignored during analysis. For example, todetermine a visit count at 2 am (i.e., of those visitors present in alocation at any time between 2 am and 3 am, in which zones were theylocated?), only those sessions including a 2 am prefixed timestamp areexamined, and a count is made for each represented zone (e.g., twovisitors at Ticketing, six visitors at security, etc.).

One example of a metric that can be determined by metrics pipeline 230is “what is the current average wait time for an individual in line forsecurity at airport space 150?” One way to evaluate the metric is formetrics pipeline 230 to examine results of the most recently completedrealtime pipeline job execution (stored in memory) for recentlycompleted sessions where visitors were in the security zone, anddetermine the average length of the sessions. Metrics for other timeperiods (e.g., “what was the average wait at 8:00 am”) can be determinedby taking the list of sessions and re-keying it by a different timeperiod. Additional examples of metrics that can be calculated in thismanner (keying on a zone, a time period, and a metric) include “how manyvisitors were seen each hour in the food court?” and “what was theaverage amount of time visitors spent in the A-gates on Tuesday?”Percentiles can also be determined using the data of platform. Forexample, “what was the 75^(th) percentile amount of time a visitor spentin the security zone on Tuesday?” or “what was the 99^(th) percentile?”

FIG. 5 illustrates an embodiment of a process for determining qualifieddevices using zone information. In various embodiments, process 500 isperformed by platform 170. The process begins at 502 when traffic dataassociated with the presence of a set of devices at a location isreceived. As one example, such traffic data is received at 502 when asensor, such as sensor 108 transmits log data (e.g., indicating that ithas observed device 110) to platform 170 via one or more networks(collectively depicted in FIG. 1A as Internet cloud 102), and that datais provided (e.g., by ELB 204) to an ingestor (e.g., ingestor 206).Portion 502 of the process may be repeated several times (e.g., withdata about the observation of device 112 also being received at 502,whether from sensor 108, or another sensor, and/or from a controller).At 504, at least some of the devices included in the set of devices arequalified as qualified devices. As one example, at 504 zoning pipeline216 evaluates data associated with the devices (e.g., by applying adecision tree of rules to log lines associated with the devices andobtained from storage 212). As another example, at 504 realtime pipeline226 evaluates data associated with the devices (e.g., by comparing thedevices against a list of banned devices). In both the cases of zoningpipeline 216 and realtime pipeline 226, at 504, those devices that arenot disqualified (i.e., survive the decision tree analysis, are not onthe banned list, or otherwise are not disqualified) are designated asqualified devices. At 506, a set of sessions associated with at leastsome of the qualified devices is created. As one example, at 506, zoningpipeline 216 determines a device-zone-duration 3-tuple for a qualifieddevice using received traffic data or a representation thereof, anexample of a session. An example of such a 3-tuple is: device 110, seenfrom 10:00 to 10:14, in ACME Clothing—First Floor. As another example,at 506, realtime pipeline 226 determines a device-zone-duration 3-tuplefor a qualified device using received traffic data or a representationthereof. An example of such a 3-tuple is: device 152, seen from 12:45 to12:59, in Airport-Terminal 1-A Gates. Finally, at 508, informationassociated with the set of sessions is provided as output. One exampleof such output being provided at 508 includes metrics pipeline 230providing metrics to either/both of Redshift 236 and Cassandra 228 (inconjunction with either the zoning pipeline or realtime pipeline, orboth, as applicable). Another example of such output being provided at508 includes the rendering or other provision of metrics to a user in aninterface, such as via interface 174 or a television screen located inairport space 150 (in communication with platform 170). The followingsection provides additional information regarding a variety ofinterfaces usable in conjunction with techniques described herein.

Zoning/Realtime Interfaces

FIG. 6 shows an interface depicting zoning information for a nationalretailer at a particular location in Boston. Interface 600 is an exampleof data that can be presented to a user (e.g., a customer representativelike Rachel) via interface 174. By clicking region 602, the user canselect a particular location in the chain. By clicking region 604, theuser can choose what time range of data to view (e.g. a particular day).By clicking region 606, the user can choose whether to see the dataacross an entire day, or by hour. As shown in FIG. 6, the entire days'worth of data is being displayed. As shown in region 608, in order toprovide a relative estimate for how busy a particular zone is at acertain time (without counts), a quartile index of Minimal, Low, Medium,High activity is used. Region 610 quantifies the percent of crossvisitation within a certain location. When the store as a whole isselected (as is the case in this view) the user sees what percentage ofall shoppers visited the different zones within a location. When acertain zone is selected, the chart will show what percentage ofshoppers that visited the selected zone also visited a different zone.Region 612 shows the breakdown of duration across all zones within alocation. When the user selects a particular zone this chart updateswith zone specific information.

FIG. 7 shows an interface depicting zoning information for a nationalretailer at a particular location in Boston. When an hour is selected(702), all data below updates.

FIG. 8 shows an interface depicting zoning information for a nationalretailer at a particular location in Boston. When a zone is selected(802), all data below updates. The level of activity is calculated, insome embodiments, by comparing the amount of traffic in a zone to ahistorical average (e.g., not relative to other zones). As shown inregion 804, a viewer of interface 800 can learn the duration breakdownof the visitors to a particular floor.

Suppose the average visitor to floor one of a store (which offershousewares) stays fifteen minutes, and an additional 25% of visitors tofloor one stay between 21 and 30 minutes. Further suppose that of thosestore visitors that visit the second floor, they stay on the floor amuch shorter time on average (e.g., stay an average of six minutes onthe second floor). If “big purchase” items (e.g., furniture) are locatedon the second floor, the comparatively short amount of time spent on thesecond floor indicates that visitors are not buying furniture.

As another example, a representative of a grocery store could use a setof interfaces similar to those shown in FIGS. 6-8 to determine howvisitors interact with different regions (defined using zones) in thestore. For example, suppose the grocery store is split into a dairy zone(at the back of the store), a middle zone (in the center of the store,where high value items are placed), and two zones (to the left and rightof the middle zone, respectively) where inexpensive items are placed.Interfaces provided by platform 170 can show how visitors interact withthose zones. For example, the grocery store may be laid out the way itcurrently is on the assumption that most shoppers need dairy items andwill take the shortest path to the dairy (i.e., go through the center ofthe store), passing by the high value items and placing some of thosehigh value items into their carts. Using techniques described herein,the store layout can be assessed, e.g., with embodiments of theinterfaces shown in FIGS. 6-8 indicating the concurrence betweenvisitors to the dairy section and each of the three other sections ofthe store, the amount of time they spend in each region, etc.

A representative of the national retailer can also use interfaces suchas those shown in FIGS. 6-8 to inform staffing and other decisions. Forexample, suppose that Monday visitor traffic to the Boston locationtypically sees the bulk of visitors staying on the first floor, withsignificantly fewer visitors visiting the second and third floors.Instead of staffing all three floors equally throughout the week,additional staff can be placed on the first floor on Mondays, with fewerstaff being placed on the second and third floors on those days.

FIG. 9 shows an interface depicting zoning information for an airport.Similar to zoning for retail spaces, zoning for airport spaces can beleveraged to view activity and duration by hour in different zones ofthe airport. Airport zoning includes arriving and departing zones.Platform 170 can identify what devices are arriving at the airport andwhat devices are departing by zone. For example, on the arrivals side,passengers typically progress from gates, passed security and/orticketing, to baggage claim. The numbers of those individuals visitingthe taxi zone vs. the limo zone vs. the rental car zone can bedetermined using techniques described herein. Determinations can also bemade about what percentage of arriving passengers stop to shop, stop forlunch, etc., in accordance with techniques described herein, and, howlong those activities take arriving passengers, on average. A departuresexample is depicted in FIG. 10.

As seen in FIG. 11, activity and duration for zoning for airports, likezoning for retail, can be viewed on an hourly basis.

As seen in FIG. 12, security areas can be used as zones and, theactivity and duration of security lines measured. The impact of theduration of time passengers spend in security lines on those passengersvisiting other areas of the airport can be evaluated using techniquesdescribed herein and interfaces such as interface 1200. For example, ifthere is a very high spike in security wait times, passengers willprobably be late for their flights, will have less time to shop/eat, andwill be going straight to the gates. And, when security lines areshorter, more co-visits through the shopping/eating zones will occur.Using techniques described herein, the impact of security lines can bequantified and visualized, allowing for more informed decisions to bemade (e.g., about staffing).

Taxi lines can also be analyzed (see FIG. 13).

FIG. 14A shows an interface for viewing line wait times at airports. Inregion 1402, users can choose what time range of duration/activity datato view for different zones. In region 1404, users can set differentthresholds to quickly identify if the wait times for a fifteen minuteperiod breached the selected threshold. In region 1406, duration isreported in fifteen minute increments. In region 1408, a depiction ofcrowding per zone is shown. FIG. 14B shows an additional security lineinterface. Taxi line wait information can similarly be seen in theinterface shown in FIG. 15.

FIG. 16A shows an interface depicting zoning information for a hotel.The activity, duration, and cross visits on an hourly basis is shown inFIG. 16A for all zones in the selected hotel. FIG. 16B shows anadditional hotel interface. Using techniques described herein andinterfaces such as interfaces 1600 and 1650, a representative of thehotel can determine which parts of the hotel are busy and when. Further,insight such as which portion of hotel restaurant visitors are notguests of the hotel can be determined (e.g., by looking at the co-visitsbetween the restaurant and areas of the hotel that only a guest wouldtypically visit (e.g., the check-in area or guest rooms). As mentionedabove, in some embodiments, a representative of a customer of platform170 (e.g., an administrator acting on behalf of a hotel) configuresplatform 170 with a list of known employee device IDs so that they canbe excluded from analysis performed by platform 170. In the context of ahotel, registering employee devices can be particularly helpful, wherehotel guests and hotel employees may have significantly more similarmovements/duration patterns than those between shoppers and retailclerks.

Additional Information Regarding Metrics

As explained above, platform 170 periodically (e.g., on hourly and dailyintervals) computes various metrics with respect to visitor data. Insome embodiments, the metrics are stored in a relational database system(RDS 242) table called “d4_metrics_tall.” The metrics can also/insteadbe stored in other locations, such as Redshift 236. The records are usedto compute metrics across various time periods per customer, zone, anddevice. A description of column names in “d4_metrics_tall” is providedbelow.

Column Name Use client_name Stores the customer name hierarchy_node_idStores the “zone” name Period Specifies if this metric is from an hourlyor daily raw log processing period_earliest The start time of theperiod. Birth The processing time of the period, or when the batchprocessing was run Metric The type of metric being calculated from theraw logs (see below) Value The calculated value of the metricconfidence_interval_low Used to specify the certainty of the andcalculated value of the metric confidence_interval_high sample_size Theamount of data processed to calculate the value of the metric

The following is a list of example metrics that can be computed byplatform 170.

Metric name Description bounce-rate The percentage of visitors who enterthe store and then leave within 2 minutes capture-rate The percentage ofdevices that meet the criteria for a visitor engagement-rate Thepercentage of visitors who enter the store and remain for at least 20minutes first-tier-dur Visits fitting within the first tier durationsecond-tier-dur Visits fitting within the second tier durationthird-tier-dur Visits fitting within the third tier durationfourth-tier-dur Visits fitting within the fourth tier durationlapsed-30-ratio The percentage of visitors who count as lapsedrecent-30-ratio The percentage of visitors counting as recentrepeat-ratio The percentage of repeat visits total-opportunity The totalnumber of visitors during the period, used to calculate other metricsvisit-duration The duration of a specific visit Visits The total numberof visits during a period Walkbys The percentage of recorded devicesthat are classified as walk-bys

Hourly Metrics:

Every hour, platform 170 calculates metrics for each zone and customeracross all data collected for the previous hour. One example hourlyreport is the hourly report by sensor (FIRES), which collates thecustomer, zone, sensor, and timestamp at which each device is seen.

Daily Metrics:

Each 24-hour period, FIRES reports are aggregated into a daily summaryby span (DSBS). This report keys metrics on a combination of customer,zone, and device. For each key, the report will collect severaltimestamps. These include the last time a device was seen as a visitor,the last time a device was seen as a walk-by, the maximum device signalstrength over the entire 24-hour period, the sum of the signal, the sumof the signal squared, the sum of the signal cubed, the event count, theinner and outer duration in seconds, and the device type. The devicetype includes but is not limited to visitor, walk-by, and access point.

Daily metrics are also calculated across all devices seen during thatday. Using previously calculated metrics, platform 170 will thencalculate a number of other statistics.

Daily metrics also include statistics covering the duration of visits.Visit length is split into distinct tiers. For example, tier 1 could beless than 5 minutes, tier 1 could be 5 to 15 minutes, and so forth. Thedaily metrics include which percentage of visitors fit into each tier ofvisit duration.

In various embodiments, aggregated daily metrics (e.g., the DSBS), arestored in RDS 242 in a table called “daily_summary_by_span”. Adescription of various fields used as a key in “daily_summary_by_span”is provided below. Other fields in the table are used to record specificmetrics and time information for specific devices in customers andzones.

Field Description the_date_local The date the record covers span_nameName of the customer zone_name Name of the zone device_id The unique IDfor the measured device manufacturer_id The unique ID used to identifythe manufacturer of the device

Platform 170 also calculates long-term metrics and presents them inreports. Among these long-term reports is a 30-day report, whichincludes the percentage of visiting devices which have been seen in azone more than once in the last 30 days, and, in some embodiments, thepercentage of lapsed visiting devices. Lapsed devices are those whichhave not visited a specific zone in 30 or more days. These percentagesare calculated per zone and included in a report that is prepared foreach customer.

Historical data is also stored and can be queried (e.g., by historicaldata parsing script, function, or other appropriate element). In variousembodiments, a query of historical data is performed against Redshift236. Results are cached in S3 (212) and read by Scala code in Spark(234). Examples of metrics that can be calculated using these resourcesinclude:

-   -   First time a device was seen in a customer's zones (across all        historical data)    -   Last time the device was seen as a visitor    -   Last time the device was seen as a walk-by    -   Maximum signal strength over the entire reporting period    -   Number of sensor observations recorded during the entire        reporting period for this device    -   Total duration of the device's visits to the zone during the        reporting period

Events

In various embodiments, platform 170 provides customers with the abilityto designate a discrete time period as an operational event, allowingfor analytics to be performed in the context of the event. An event canbe an arbitrary designation of a date range (e.g., “March 2016” and canalso correspond to promotional or other events (e.g., “SpringClearance”). The following are examples of scenarios in which eventsmight be created within platform 170:

-   -   An analytics manager from a fast casual restaurant can enter the        dates and expected revenue from a recent promotion to understand        if offers/menu items drew the expected results. The analytics        manager might share the information with marketing colleagues to        influence future campaigns, in addition to the necessary        leadership as part of a reporting exercise.    -   A regional operations manager at a mid-sized specialty retailer        can use an event to understand the effectiveness of a training        program on his team's ability to engage customers. For example,        suppose the manager has noticed a declining engagement rate        month-on-month. The manager can use eventing to understand if        the new educational program drew his expected engagement result        and further had an impact on sales in his stores during a        particular period.    -   A marketing campaign manager from a national bank chain is        responsible for driving new visitor traffic into the new        bank-cafe hybrid locations. The locations serve coffee and tea        but not food. The manager can use eventing to compare the        performance of different food vendors. For example, the manager        could run a campaign with a waffle company one week and then a        scone vendor a few weeks later. Using eventing, the manager can        leverage AB testing to select the better long-term food partner        in encouraging storefront conversion and new visitor traffic.

In the following example, suppose Rachel has been tasked with creatingan event and evaluating visitor traffic associated with the event. Asample interface for creating an event is shown in FIG. 17 (and is anexample of an interface that can be provided by platform 170, e.g., viainterface 174). An alternate interface for initiating the creation of anevent is shown in FIG. 18. To create a new event, Rachel clicks onregion 1702 (or region 1802, as applicable). After doing so, Alice ispresented with the interface shown in FIG. 19, where she is asked topick a type of event. Suppose Alice picks “Marketing Campaign” byselecting region 1902. She is presented with the interface shown in FIG.20 in response and prompted to supply various information with respectto event creation. Note that an event can be created retroactively. Forexample, Alice can create a “Winter Markdown” event for ACME on platform170 even after the date range specified for the event has ended,allowing for retroactive analysis of data pertinent during the specifieddate range.

In particular, in the interface shown in FIG. 20, Alice is prompted tocreate an event by adding an event name, event description, location(whether an individual location or hierarchy level), date range for theevent, and (optionally) expected sales for the event.

Once the event is created (and has commenced), Alice can view theperformance of the event in a summary page interface, an embodiment ofwhich is shown in FIG. 21. From the summary page interface, Alice canselect specific locations, update the comparison period, edit the event,create a new event, and view upcoming events.

The summary page interface includes a metrics box 2102. In the exampleshown in FIG. 21, “storefront conversion” indicates how effective alocation was at getting visitors into the location. “Traffic count” iscount of visitors. “Bounce rate” indicates the number of visitors wholeft within five minutes.

Visitor Profile

An alternate embodiment of a summary page interface is shown in FIG. 22.The summary page shown in FIG. 22 includes a visitor profile section2202. The visitor profile provides Alice with an understanding of thetype of customers entering a location during an event. In particular,the summary includes three kinds of evaluations: Frequency During Event(2204), Returning Visitors (2206), and Other Events Visited (2208). Eachsection provides a different view into the loyalty profile of the eventvisitors.

The event frequency (2204) is the ratio of visitors who are recorded atan event across distinct segments of time. For example, an event lastingthree days might have event frequencies measured in 1-day increments. Anevent frequency report in such a scenario would indicate that a certainnumber of visitors were recorded during only one total day of the event,a smaller number during two separate days of the event, and an evensmaller number during all three days of the event. An event frequencyreport can also include the total sample size or number of devicesrecorded during the event. In various embodiments, event frequencyreports are stored in S3 or another appropriate location, allowingmultiple events to be compared using multiple event frequency reports.When an event frequency report is generated (e.g., from a database), itis given a birth timestamp, which is the time at which the report wasoriginally created. An event frequency report can also specify thebeginning and end times of the event. In the example shown in FIG. 22,Alice can hover over each bar in region 2204 to see actual frequencyvalues. Frequency metrics can also be determined outside of specificevents, as applicable. For example, a fast food restaurant may choose toset an arbitrary time period (e.g., a week or a month) and measure on arecurring basis (e.g., with a histogram similar to that depicted in2204) the number of visits made by customers in that time period.

The return rate (also referred to herein as “revisitation”) of visitorsafter an event has concluded is depicted in region 2206. In variousembodiments, event revisitation data is kept in a table in RDS 242called “d4_event_revisitation.” A returning visitors report can be runat any time after the conclusion of an event, and reports on thepercentage of visitors seen during an event who have been recorded in acustomer's zones for the first time since the end of the event.Percentages are reported over 24-hour periods. The maximum timespancovered by the report is determined by the lesser of two values: (1) thelength of time at which 100% of visitors seen during the event have beenrecorded in a customer's zones since the conclusion of the event, and(2) a configurable time period that defaults to six months. Alice canhover over each point in the graph shown in region 2206 to see actualvalues.

Depicted in region 2208 is an indication of other events visited byvisitors to the instant event (e.g., at the instant location). Thereport includes the percentage of visitors who were present during eachevent in the report compared to the total number of distinct visitors toall events in the report. One way to determine metrics on which deviceshave been to which (multiple) events is to tag records associated withdevices the event identifiers. Another way to determine “other eventsvisited” metrics (e.g., as shown in region 2208) is as follows. Eachevent at a given location has associated with it event metadata. A givenevent has a start date and an end date. All of the devices observedwithin the start/end date of a first event can then be checked todetermine whether they were also observed within the start/end date ofeach of the other events (e.g., a comparison against the dates of thesecond event, a comparison against the dates of the third event, etc.).The results are ranked and the events with the highest amount ofoverlapping observed devices are presented in region 2208.

The following are examples of scenarios in which data in the visitorprofile is used by a representative of a customer of platform 170:

-   -   The analytics manager from the fast casual restaurant can use        the visitor profile to understand if a recent menu promotion        encouraged repeat visits during the allotted time that the        promotion ran. With that information, the manager can start to        compare events and opt to plan future promotions based on the        stickiness of past ones.    -   Suppose the regional operations manager at the mid-sized        specialty retailer has rolled out a new training to his staff in        which they create closer relationships with customers and        sometimes seek their contact information for follow-up. The        manager can use the visitor profile to see if this tactic is        effective at encouraging an increase in repeat visitors over        time, signaling that loyalty is being nurtured by his staff.    -   Suppose a marketing campaign manager for a national pet        food/supplies chain has been urging management to pull back from        doing discount-driven promotions, as she suspects that such        promotions do not attract valuable customers for the chain. The        manager could test two promotions: one that is discount-driven        (e.g., 20% off all pet bedding) and one that is not (e.g.,        “check out our new indestructible chew toys”). With the        discount-driven promotion, she will be able to tell if the        overlap with other events confirms her suspicion about a        customer segment that only visits during discounts. Furthermore,        she might be able to tell which promotion encourages more repeat        visits after the conclusion of the event.

Visitor Loyalty Behavior

Also included in interface 2200 is region 2210, which indicates visitorloyalty behavior. In particular, region 2210 reports on the percentageof customers who are new (2212), re-engaged (2214), or recent (2216). Inaddition to the current breakdown of visitor types (49.2% new; 19.8%re-engaged; 29.9% recent), a comparison between the current breakdownand a previous time period (e.g., a previous event) is included (i.e.,−3.6%; −0.5%; 3.2%).

A new visitor is one who has not been seen previously (e.g., at thereporting location, or at any location, as applicable). A visitor willremain classified as new until he returns to a previously visitedlocation. A re-engaged visitor is one who has visited the same locationat least twice, and whose last visit to that location was more than 30days ago. In various embodiments, 30 days is used as a default thresholdvalue. The value is customizable. For example, certain types ofbusinesses (e.g., oil change facilities) may choose to use a longerduration (e.g. 60 or 90 days) to better align with their naturalcustomer cycle, whereas other businesses (e.g., coffee shops) may chooseto use a shorter duration (e.g., 14 days). A recent visitor is one hasvisited the same location at least twice, and whose previous visit waswithin the last 30 days.

An alternate embodiment of an interface depicting loyalty information isshown in FIG. 23 (in region 2302).

The following are examples of scenarios in which a user of platform 170is interested in the ability to differentiate between kinds of visitorloyalty behavior:

-   -   Sean is responsible for regional merchandising for a national        retail chain for teens. He currently plans for a large shipment        every 30 days. Knowing that his more loyal customers visit that        frequently, he configures the chain's account with platform 170        such that a “recent” shopper is one who visits every 30 days.        Using the “re-engaged” metric, Sean will be able to see if a        certain month's merchandise is more effective at bringing in        customers who may be slipping away. Similarly, should he choose        to push the merchandise with an in-store event or advertising,        he may be able to observe whether the additional marketing spend        increased the “re-engaged” metric with the end goal of moving        “re-engaged” customers into the “recent” bucket.    -   Jenn manages marketing campaigns for a regional coffee and tea        chain. She knows that her Fall menu typically drives increased        traffic into the locations, particularly from non-regular        customers. This year, she would like to see if she can bring        those less loyal customers in before the seasonal items are        introduced, and also see if she can keep them longer. One option        she has is to start promotion early and track the success        through the “re-engaged.” Once the Fall menu is formally        introduced she can compare the subsequent “re-engaged” metric to        the one observed after her early promotion kicks off. An example        of performing a comparison between two periods' re-engaged        metrics is shown in FIG. 24. Over the course of the Fall season,        Jenn can also track the “new” visitor number closely (e.g., to        ensure it has decreased steadily but not too much).

In various embodiments, the interface provided to a user of platform 170is configurable by that user. For example, a user can indicate whichwidgets should be presented to the user in a dashboard view. In theinterface shown in FIG. 25, the user is reviewing options for includingvisitor loyalty data in the dashboard view.

FIG. 26 illustrates an embodiment of a process for assessing visitorcomposition. In various embodiments, process 2600 is performed byplatform 170. The process begins at 2602 when traffic data associatedwith the presence of a set of devices at a location is received. As oneexample, such traffic data is received at 2602 when a sensor, such assensor 108 transmits log data (e.g., indicating that it has observeddevice 110) to platform 170 via one or more networks (collectivelydepicted in FIG. 1A as Internet cloud 102), and that data is provided(e.g., by ELB 204) to an ingestor (e.g., ingestor 206). Portion 2602 ofthe process may be repeated several times (e.g., with data about theobservation of device 112 also being received at 2602, whether fromsensor 108, or another sensor, and/or from a controller). At 2604, thedevices are segmented based on a status. Examples of device statusinclude (for a given location) whether the device is “new,”“re-engaged,” or “recent.” In various embodiments, segmentation isperformed by metrics pipeline 230 (described in more detail above)evaluating log data (e.g., in storage 212, RDS 242, Redshift 236, and/orCassandra 228) as applicable and annotating the log data in accordancewith rules such as those provided above (i.e., using the definitions ofnew/re-engaged/recent visitors). At 2606, data associated with thesegmentation is provided as output. As one example, a breakdown ofvisitor composition is depicted (e.g., at 2606) in the interface shownin FIG. 22 in region 2210. As shown in FIG. 22, the view presented ininterface 2200 is dynamic, and portion 2606 can be repeated (e.g., inresponse to user interactions with interface 2200).

Events Pipeline Wrapper

Events pipeline wrapper 240 (eventsPipelineWrapper.py) is a Pythonscript that calculates events-based metrics in various embodiments. Inparticular, events pipeline wrapper 240 outputs the following: (1) eventfrequency; (2) revisitation; and (3) overlap. FIGS. 27-30 collectivelydepict an example implementation of an events pipeline wrapper script.

In various embodiments, an RDS table called “d4 event frequency” (keyedby customer, zone, an event identifier, and start/end times) is includesthe following fields:

Field Description client_name The customer name hierarchy_node_id Thezone name start_date The beginning of the event end_date The end of theevent Birth The time at which the metric was calculated Metric Themetric calculated (visitor-frequency) frequency_level The number of daysfor which visitor frequency was calculated Value The count of distinctvisitors detected by the zone's sensors for the number of days in the“frequency_level” column sample_size The total number of visitorsdetected by the zone's sensors over the entire duration of the event.

Sample data from the “d4 event frequency” table is shown in FIG. 31. Inthe example of FIG. 31, a three day event was held. A total of 4616unique devices were seen at sensor 112_L-11 during the three day event.Of those devices, 4549 visited once, 63 visited two of the three days,and 4 visited all three days. A total of 1489 unique devices were seenat sensor 161_TE2. Of those devices, 1474 visited once, 15 visited twoof the three days, and no devices visited all three days.

FIG. 32 illustrates an embodiment of a process for determining co-visitsby visitors. In various embodiments, process 3200 is performed byplatform 170. The process begins at 3202 when traffic data associatedwith the presence of a set of devices at a location is received. As oneexample, such traffic data is received at 3202 when a sensor, such assensor 108 transmits log data (e.g., indicating that it has observeddevice 110) to platform 170 via one or more networks (collectivelydepicted in FIG. 1A as Internet cloud 102), and that data is provided(e.g., by ELB 204) to an ingestor (e.g., ingestor 206). Portion 3202 ofthe process may be repeated several times (e.g., with data about theobservation of device 112 also being received at 3202, whether fromsensor 108, or another sensor, and/or from a controller). At 3204, adetermination is made that a first device was present at a firstlocation at a first time (e.g., during an event). In variousembodiments, the determination is made by events pipeline wrapper 240.At 3206, a determination is made that the device was also present at thefirst location at a second time (e.g., during a subsequent event). Invarious embodiments, the determination is also made by events pipelinewrapper 240. In various embodiments, portions 3204 and/or 3206 ofprocess 3200 are performed by metrics pipeline 230 (described in moredetail above) evaluating log data (e.g., in storage 212, RDS 242,Redshift 236, and/or Cassandra 228) as applicable and annotating the logdata. Finally, at 3208, data associated with the co-visit (of the deviceto the first location on two different occasions) is provided as output.As one example, a breakdown of visitor co-visits is depicted (e.g., at2608) in the interface shown in FIG. 22 in region 2202. Additionaldiscussion of aspects of process 3200 are provided above (e.g., inconjunction with discussion of FIG. 22).

FIG. 33 illustrates an embodiment of a process for determiningre-visitation by visitors. In various embodiments, process 3300 isperformed by platform 170. The process begins at 3302 when traffic dataassociated with the presence of a set of devices at a location isreceived. As one example, such traffic data is received at 3302 when asensor, such as sensor 108 transmits log data (e.g., indicating that ithas observed device 110) to platform 170 via one or more networks(collectively depicted in FIG. 1A as Internet cloud 102), and that datais provided (e.g., by ELB 204) to an ingestor (e.g., ingestor 206).Portion 3302 of the process may be repeated several times (e.g., withdata about the observation of device 112 also being received at 3302,whether from sensor 108, or another sensor, and/or from a controller).At 3304, a determination is made that a first device was present at afirst location at a first time (e.g., during an event). In variousembodiments, the determination is made by events pipeline wrapper 240.At 3306, a determination is made that the device was also present at thefirst location at a second time (e.g., at a time subsequent to theevent). In various embodiments, the determination is also made by eventspipeline wrapper 240. In various embodiments, portions 3304 and/or 3306of process 3300 are performed by metrics pipeline 230 (described in moredetail above) evaluating log data (e.g., in storage 212, RDS 242,Redshift 236, and/or Cassandra 228) as applicable and annotating the logdata. Finally, at 3308, data associated with the re-visit (of the deviceto the first location at a subsequent time) is provided as output. Asone example, a breakdown of the lengths of time it took for visitors tore-visit is depicted (e.g., at 2606) in the interface shown in FIG. 22in region 2202. Additional discussion of aspects of process 3300 areprovided above (e.g., in conjunction with discussion of FIG. 22).

FIG. 34 illustrates an embodiment of a process for assessing visitorfrequency during an event. In various embodiments, process 3400 isperformed by platform 170. The process begins at 3402 when traffic dataassociated with the presence of a set of devices at a location isreceived. As one example, such traffic data is received at 3402 when asensor, such as sensor 108 transmits log data (e.g., indicating that ithas observed device 110) to platform 170 via one or more networks(collectively depicted in FIG. 1A as Internet cloud 102), and that datais provided (e.g., by ELB 204) to an ingestor (e.g., ingestor 206).Portion 3402 of the process may be repeated several times (e.g., withdata about the observation of device 112 also being received at 3402,whether from sensor 108, or another sensor, and/or from a controller).At 3404, a determination is made of the frequency of the number of timesthat a given device was observed at the location. In variousembodiments, the frequency analysis is performed by events pipelinewrapper 240. In various embodiments, the frequency analysis is performedby metrics pipeline 230 (described in more detail above) evaluating logdata (e.g., in storage 212, RDS 242, Redshift 236, and/or Cassandra 228)as applicable and annotating the log data. At 3406, data associated withthe frequency is provided as output. As one example, a breakdown ofvisitor frequency is depicted (e.g., at 3406) in the interface shown inFIG. 22 in region 2204. Additional discussion of aspects of process 3400are provided above (e.g., in conjunction with discussion of FIG. 22).

Although the foregoing embodiments have been described in some detailfor purposes of clarity of understanding, the invention is not limitedto the details provided. There are many alternative ways of implementingthe invention. The disclosed embodiments are illustrative and notrestrictive.

What is claimed is: 1.-20. (canceled)
 21. A system comprising: at leastone processor; and at least one computer readable storage medium,excluding transitory signals, coupled to the at least one processor, andstoring executable computer program instructions, wherein the computerprogram instructions, when executed by the at least one processor, causethe processor to perform steps comprising: receiving traffic dataassociated with physical presence of a set of devices at a physicallocation, wherein the received traffic data includes log data associatedwith the physical presence of the set of devices received from a sensorat a zone associated with the physical location; based at least in parton the traffic data, determining a duration of time that a given deviceincluded in the set of devices has been physically present at thephysical location; classifying at least some of the devices included inthe set of devices as qualified devices when the determined duration oftime that the given device has been physically present at the physicallocation is between a threshold minimum amount of time and a thresholdmaximum amount of time; creating a set of sessions associated with oneor more of the qualified devices, wherein creating sessions includesdetermining a device-zone-duration 3-tuple for each of the one or morequalified devices satisfies at least two criteria for creating the setof sessions; and outputting information associated with the created setof sessions, wherein the information includes a set of metricsindicating a count of qualified devices located at the zone for each ofmultiple periods of time.
 22. The system of claim 21, wherein thetraffic data includes log data received from a first sensor at a firstzone associated with the physical location and log data received from asecond sensor at a second zone associated with the physical location,and wherein the set of metrics further includes an indication of a countof qualified devices that were physically located at both the first zoneand the second zone.
 23. The system of claim 21, wherein the trafficdata includes log data received from a first sensor at a first zoneassociated with the physical location and log data received a secondsensor at a second zone associated with the physical location, andwherein the set of metrics further includes a comparison betweendurations of time qualified devices were physically present at the firstzone and durations of time qualified devices were physically present atthe second zone.
 24. The system of claim 21, wherein classifying atleast some of the devices included in the set of devices as qualifieddevices includes qualifying devices based on a combination of thedetermined duration of time for the qualified devices and a receivedsignal strength indicator (RSSI) for the qualified devices.
 25. Thesystem of claim 21, wherein the device-zone-duration 3-tuple mapsinformation identifying the given device to information identifying thezone associated with the physical location to information identifying aduration of the given device within the zone.
 26. The system of claim21, wherein the processor further performs steps comprising adding thegiven device included in the set of devices to a blacklist of devicesbased at least in part on a determination that the given device wasobserved for a duration that exceeds the threshold maximum amount oftime.
 27. The system of claim 21, wherein the physical location is abuilding, and wherein the zone is a floor of the building.
 28. A methodcomprising: receiving traffic data associated with physical presence ofa set of devices at a physical location, wherein the traffic dataincludes log data received from sensors at multiple zones associatedwith the physical location; for a device in the set of devices,detecting, based on the traffic data, a duration of time the device hasbeen physically present at the physical location and an identifier ofthe zone; creating sessions associated with multiple devices in the setof devices, wherein creating sessions includes determining whether theduration of time and the identifier of the zone detected for one of themultiple devices satisfy one or more criteria for creating the sessions;and outputting a set of metrics associated with the created sessions,wherein the metrics include an indication of a count of devices thatwere physically located at both a first zone associated with thephysical location and a second, different zone associated with thephysical location.
 29. The method of claim 28, wherein the set ofmetrics further indicate a count of the devices located at the firstzone or the second zone for each of multiple periods of time.
 30. Themethod of claim 28, wherein the set of metrics further includes acomparison between durations of time devices were physically present atthe first zone and durations of time devices were physically present atthe second zone.
 31. The method of claim 28, further comprisingclassifying at least some of the devices in the set of devices asqualified devices based on a combination of the determined duration oftime for the qualified devices and a received signal strength indicator(RSSI) for the qualified devices, wherein the sessions are created forthe qualified devices.
 32. The method of claim 28, wherein the physicallocation is a building, wherein the first zone is a first floor of thebuilding and the second zone is a second floor of the building.
 33. Acomputer readable storage medium, excluding transitory signals, andstoring executable computer program instructions that when executed by aprocessor cause the processor to perform steps comprising: receivingtraffic data indicating physical presence of a set of devices at aphysical location from a sensor at a zone associated with the physicallocation; classifying a device in the set of devices as a qualifieddevice if a duration of time the device has been physically present atthe physical location is between a threshold minimum amount of time anda threshold maximum amount of time; for each of multiple qualifieddevices, creating a set of sessions associated with the qualified deviceupon determining that a device-zone-duration tuple for the qualifieddevice satisfies at least two criteria for creating the set of sessions;and generating metrics for the created set of sessions based on theduration of time a respective qualified device has been physicallypresent at the physical location.
 34. The non-transitory computerreadable storage medium of claim 33, wherein the metrics indicate acount of qualified devices located at the zone for each of multipleperiods of time.
 35. The non-transitory computer readable storage mediumof claim 33, wherein the traffic data includes log data received from afirst sensor at a first zone associated with the physical location andlog data received from a second sensor at a second zone associated withthe physical location, and wherein the set of metrics further includesan indication of a count of qualified devices that were physicallylocated at both the first zone and the second zone.
 36. Thenon-transitory computer readable storage medium of claim 33, wherein thetraffic data includes log data received from a first sensor at a firstzone associated with the physical location and log data received asecond sensor at a second zone associated with the physical location,and wherein the set of metrics further includes a comparison betweendurations of time qualified devices were physically present at the firstzone and durations of time qualified devices were physically present atthe second zone.
 37. The non-transitory computer readable storage mediumof claim 33, wherein classifying the device in the set of devices as aqualified device includes qualifying the device based on a combinationof the determined duration of time for the device and a received signalstrength indicator (RSSI) for the device.
 38. The non-transitorycomputer readable storage medium of claim 33, wherein thedevice-zone-duration 3-tuple maps information identifying the qualifieddevice to information identifying the zone associated with the physicallocation to information identifying a duration of the qualified devicewithin the zone.
 39. The non-transitory computer readable storage mediumof claim 33, wherein the steps further comprising adding a selecteddevice in the set of devices to a blacklist of devices based at least inpart on a determination that the selected device was observed for aduration that exceeds the threshold maximum amount of time.
 40. Thenon-transitory computer readable storage medium of claim 33, wherein thephysical location is a building, and wherein the zone is a floor of thebuilding.